The present invention relates to a data processing system comprising a control system and a weight factor generator considering automated forms of data processing, more particularly implementing forms of errors-in-variables data reductions being rendered to include essential weighting of squared reduction deviations in order to provide for adequate representation of system behavior.
As empirical relationships are often required to describe system behavior, data analysts continue to rely upon least-squares and maximum likelihood approximation methods to fit both linear and nonlinear functions to experimental data. Fundamental concepts, related to both maximum likelihood estimating and least-squares curve fitting, stem from the early practice referred to in 1766 by Euler as calculus of variation. The related concepts were developed in the mid 1700's, primarily through the efforts of Lagrange and Euler, utilizing operations of calculus for locating maximum and minimum function value correspondence. The maximum and minimum values and certain inflection points of the function occur at coordinates which correspond to points of zero slope along the curve. To determine the point where a minimum or maximum occurs, one derives an expression for the derivative (or slope) of the function and equates the expression to zero. By merely equating the derivative of the function to zero, local parameters, which respectively establish the maximum or minimum function values, can be determined.
The process of Least-Squares analysis utilizes a form of calculus of variation in statistical application to determine fitting parameters which establish a minimum value for the sum of squared single component residual deviations from a parametric fitting function. The process was first publicized in 1805 by Legendre. Actual invention of the least-squares method is clearly credited to Gauss, who as a teenage prodigy first developed and utilized it prior to his entrance into the University of Göttingen.
Maximum likelihood estimating has a somewhat more general application than that of least-squares analysis. It is traditionally based upon the concept of maximizing a likelihood, which may be defined either as the product of discrete sample probabilities or as the product of measurement sample probability densities, for the current analogy and in accordance with the present invention, it may be either, or a combination of both. By far, the most commonly considered form for representing a probability density function is referred to as the normal probability density distribution function (or Gaussian distribution). The respective Gaussian probability density function as formulated for a standard deviation of σYYin the measurement of  will take the form of Equation 1:
                                          D            ⁡                          (                              Y                -                y                            )                                =                                    1                                                2                  ⁢                  π                  ⁢                                                                          ⁢                                      σ                    Y                    2                                                                        ⁢                          ⅇ                              -                                                                            (                                              Y                        -                        y                                            )                                        2                                                        2                    ⁢                                          σ                      Y                      2                                                                                                          ,                            (        1        )            wherein D represents a probability density, Y represents either a single component observation or a dependent variable measurement, and  represents the expected or true value for said single component or said dependent variable. The formula for the Gaussian distribution was apparently derived by Abraham de Moivre in about 1733. The distribution function is dubbed Gaussian distribution due to extensive efforts of Gauss related to characterization of observable errors. Consistent with the concept of a probability density distribution function, the actual probability of occurrence is considered as the integral or sum of the probability density, taken (or summed) over a range of possible samples. A characteristic of probability distribution functions is that the area under the curve, considered between minus and plus infinity or over the range of all possible dependent variable measurements, will always be equal to unity. Thus, the probability of any arbitrary sample lying within the range of the distribution function entire is one, e.g.,
                                          ∫                          -              ∞                                      +              ∞                                ⁢                                    D              ⁡                              (                                  Y                  -                  y                                )                                      ⁢                          ⅆ              Y                                      =        1.                            (        2        )            For a typical linear Gaussian Likelihood estimator, LY, being considered to exemplify variations in the measurement of  as a single valued function or as a linear function with the mean squared deviations associated with each data sample being independent of coordinate location, the explicit likelihood estimator will take the form of Equation 3:
                                                                        L                Y                            =                                                ∏                                      k                    =                    1                                    K                                ⁢                                                                  ⁢                                                      1                                                                  2                        ⁢                        π                        ⁢                                                                                                  ⁢                                                  σ                          Y                          2                                                                                                      ⁢                                      ⅇ                                          -                                                                                                    (                                                          Y                              -                              y                                                        )                                                    k                          2                                                                          2                          ⁢                                                      σ                            Y                            2                                                                                                                                                                                                                      =                                                (                                                            ∏                                              k                        =                        1                                            K                                        ⁢                                                                                  ⁢                                          1                                                                        2                          ⁢                          π                          ⁢                                                                                                          ⁢                                                      σ                            Y                            2                                                                                                                                )                                ⁢                                                      ⅇ                                          -                                                                        ∑                                                      k                            =                            1                                                    K                                                ⁢                                                                                                  ⁢                                                                                                            (                                                              Y                                -                                y                                                            )                                                        k                            2                                                                                2                            ⁢                                                          σ                              Y                              2                                                                                                                                                            .                                                                                        (        3        )            The Y subscript on the likelihood estimator without an additional subscript indicates the product of probabilities (or the product of probability density functions) being related to measurements of the dependent variable, , as an analytical representation of a respective data sample, Yk. The lower case italic  subscript designates the data sample or respective data-point coordinate measurement, and the upper case K represents the total number of data points being considered.
A simplified form for maximizing the likelihood is rendered by taking the natural log of the estimator, as exemplified by Equation 4:
                              ln          ⁢                                          ⁢                      L            Y                          =                              ln            (                                          ∏                                  k                  =                  1                                K                            ⁢                                                          ⁢                              1                                                      2                    ⁢                                          πσ                      Y                      2                                                                                            )                    -                                    ∑                              k                =                1                            K                        ⁢                                                  ⁢                                                                                (                                          Y                      -                      y                                        )                                    k                  2                                                  2                  ⁢                                      σ                    Y                    2                                                              .                                                          (        4        )            Since the maximum values for the natural log of LY will always coincide with the maximum values for LY, maximum likelihood can be determined by equating the derivatives of ln LY to zero. The first term on the right hand side of Equation 4 can be considered to be a determined constant which need not be included. The term on the far right represents minus one half of the respective sum of squared deviations, so that maximizing the log of the likelihood should provide the same set of inversion equations as will minimize the respective sum of correspondingly weighted square deviations. In accordance with the present invention, the likelihood estimator is independent of the sign of a deviation being squared, so that whether the deviation is generated as Y- or -Y, the square of that deviation will be the same. Taking the partial derivative of ln LY with respect to each of the fitting parameters, Pp, will yield:
                                                        ∂              ln                        ⁢                                                  ⁢                          L              Y                                            ∂                          P              p                                      =                              ∑                          k              =              1                        K                    ⁢                                          ⁢                                                                      (                                      Y                    -                    y                                    )                                k                                            σ                Y                2                                      ⁢                                                            ∂                                      y                    k                                                                    ∂                                      P                    p                                                              .                                                          (        5        )            The p subscript is included to respectively designate each included fitting parameter. Replacing the parametric fitting parameter representations, Pp, by determined ones, p, and equating the partial derivatives to zero will yield Equations 6:
                                          ∑                          k              =              1                        K                    ⁢                                          ⁢                                                                      (                                      Y                    -                    y                                    )                                k                                            σ                Y                2                                      ⁢                                          (                                                      ∂                                          y                      k                                                                            ∂                                          P                      p                                                                      )                                            𝒫                p                                                    =        0.                            (        6        )            The close parenthesis with double subscript p is included to indicate replacement of each Pp with its respectively determined counter part, p. The k subscript infers representation of, or evaluation with respect to, a corresponding observation sample measurement or a respective coordinate sample datum.
Note that the construction of the center equality of Equation 3 is based upon the assumption that the likely deviation of each included sample is Gaussian. Such is seldom the case, but the validity of Equation 3 can be alternately based upon the premise that the sums of arbitrary groupings of sample deviations with non-skewed uncertainty distributions may also be considered as Gaussian.
In accordance with the present invention, non-skewed error distributions, including non-skewed probability density distributions, may be defined as any form of observation uncertainty distributions for which the mean sample value can always be assumed to approach a “true” value (or acceptably accurate mean representation for what is assumed to be the expected or true value) in the limit as the number of random samples approaches infinity.
In accordance with the present invention, mean squared deviations, which are established from groupings of arbitrary samples of non-skewed homogeneous error distributions, can be treated as Gaussian. By alternately considering the likelihood estimator as the product of probabilities of one or more such groupings, rather than the product of individual sampling probabilities, the validity of Equation 3 may be established. In accordance with the present invention, the validity of Equation 3 may be established for applications which are subject to the condition that the summation in the exponent of the second term on the right is at least locally representative of sufficient numbers of data samples of non-skewed uncertainty distribution to establish appropriate mean values along the fitting function. The likelihood estimator can be alternately written in the form of Equations 7 to establish representation of such groupings:
                                                                        L                Y                            =                                                ∏                                      g                    =                    1                                    G                                ⁢                                                                  ⁢                                                      ∏                                                                  k                        g                                            =                      1                                                              K                      g                                                        ⁢                                                                          ⁢                                                            1                                                                        2                          ⁢                                                      πσ                            Y                            2                                                                                                                ⁢                                          ⅇ                                              -                                                                              ∑                                                                                          k                                g                                                            =                              1                                                                                      K                              g                                                                                ⁢                                                                                                          ⁢                                                                                                                    (                                                                                                      Y                                    g                                                                    -                                                                      y                                    g                                                                                                  )                                                                                            k                                g                                                            2                                                                                      2                              ⁢                                                              σ                                Y                                2                                                                                                                                                                                                                                                                                =                                                ∏                                      g                    =                    1                                    G                                ⁢                                                                  ⁢                                                      ∏                                                                  k                        g                                            =                      1                                                              K                      g                                                        ⁢                                                                          ⁢                                                            1                                                                        2                          ⁢                                                      πσ                            Y                            2                                                                                                                ⁢                                          ⅇ                                              -                                                                                                            K                              g                                                        ⁢                                                                                                                            (                                                                                                            Y                                      g                                                                        -                                                                          y                                      g                                                                                                        )                                                                2                                                            _                                                                                                            2                            ⁢                                                          σ                              Y                              2                                                                                                                                                                                                                                                  =                                                (                                                            ∏                                              g                        =                        1                                            G                                        ⁢                                                                                  ⁢                                                                  ∏                                                                              k                            g                                                    =                          1                                                                          K                          g                                                                    ⁢                                                                                          ⁢                                              1                                                                              2                            ⁢                                                          πσ                              Y                              2                                                                                                                                                            )                                ⁢                                  ⅇ                                      -                                                                  ∑                                                  g                          =                          1                                                G                                            ⁢                                                                                          ⁢                                                                                                    K                            g                                                    ⁢                                                                                                                    (                                                                                                      Y                                    g                                                                    -                                                                      y                                    g                                                                                                  )                                                            2                                                        _                                                                                                    2                          ⁢                                                      σ                            Y                            2                                                                                                                                                                                                                      =                                                (                                                            ∏                                              k                        =                        1                                            K                                        ⁢                                                                                  ⁢                                          1                                                                        2                          ⁢                          π                          ⁢                                                                                                          ⁢                                                      σ                            Y                            2                                                                                                                                )                                ⁢                                                      ⅇ                                          -                                                                        ∑                                                      k                            =                            1                                                    K                                                ⁢                                                                                                  ⁢                                                                                                            (                                                              Y                                -                                y                                                            )                                                        k                            2                                                                                2                            ⁢                                                          σ                              Y                              2                                                                                                                                                            .                                                                                        (        7        )            The subscript g of Equations 7 designates the group; the typewriter type G represents the number of groups; the Kg represents the number of samples associated with each respective group; and the kg refers to the specific sample of the respective group, such that the total number of data samples is equal to the sum of the samples included in each group. The line over the squared deviations is placed to indicate the mean squared deviation which may be statistically considered or simply obtained by dividing the sum of the squared deviations by the number of addends, or in this example Kg. Notice that a relative weighting of the mean squared deviation of each group, as included in the overall sum of squared deviations, is dependent upon an observation occurrence which, in this example, may be assumed to be proportional to the number of elements in the respective group and not the square of said number of elements. In addition, in accordance with the present invention, note that changes in slope along a fitting function segment will also affect probability of occurrence. The terminology “locally representative,” as considered in correspondence with a specified fitting function, may be defined as over local regions with only small or assumed insignificant changes in slope, or said locally representative may be alternately defined as over local regions without extreme changes in slope.
In consideration of applications of Equation 3, with provision of sample groupings as exemplified by Equations 7 being subject to the condition that the mean square deviations of each of the considered groupings can be assumed to be representative of a Gaussian distribution, in accordance with the present invention the validity of Equations 6 can be established in any one of three ways. These are:
1. Each data sample can be representative of a uniform Gaussian uncertainty distribution over the extremities of a linear fitting function;
2. Each data sample can be representative of a point-wise non-skewed uncertainty distribution, assuming sufficient data samples of a same distribution are provided at each localized region along the fitting function to establish localized sums of nonlinear samples as being characterized by homogeneous Gaussian distribution functions;3. Each data sample can be representative of a point-wise Gaussian uncertainty distribution, also assuming sufficient data samples of a same distribution are provided at each localized region along the fitting function to establish localized sums of nonlinear samples as being characterized by homogeneous Gaussian distribution functions.In accordance with the present invention, conditions for maximum likelihood can be alternately realized for data not satisfying any of these three criteria, provided that the elements of the likelihood estimator, as rendered to represent the observation samples and as correspondingly rendered in the sum of squared reduction deviations can be appropriately rendered, normalized, and weighted to compensate for skewed error distributions, nonlinearities, and all associated heterogeneous sampling. In accordance with the present invention, reduction deviations are analytically represented deviations that are assumed to characterize a mean error displacement path. Reduction deviations, alternately referred to herein as path oriented deviations, can be rendered in any of at least four representative forms. These are:1. coordinate oriented residual deviations,2. coordinate oriented data-point projections,3. path coincident deviations, and4. path oriented data-point projections.
The sums of squared reduction deviation as included in representing the likelihood elements should be rendered to weight all considered forms of observation occurrence. In accordance with the present invention, tailored weight factors should be included within said sums of squared reduction deviations to compensate for observation occurrence which may be associated with nonlinear and heterogeneous observation sampling, thus allowing each individual representation of path coincident deviation or data-point projection which might be included in the likelihood estimator to be characterized by a single, unified, and normal (or Gaussian) uncertainty distribution.
In accordance with the present invention, Equations 6 may be alternately written to compensate for skewed uncertainty distributions, nonlinearities and/or heterogeneous sampling by including representation for an essential weight factor, W, as in Equations 8:
                                          ∑                          k              =              1                        K                    ⁢                                          ⁢                                                                      𝒲                                      Y                    k                                                  ⁡                                  (                                      Y                    -                    y                                    )                                            k                        ⁢                                          (                                                      ∂                                          y                      k                                                                            ∂                                          P                      p                                                                      )                                            𝒫                p                                                    =        0.                            (        8        )            The Y subscript on the essential weight factor, as in the case of Equations 8, implies the weighting of residual deviations between dependent variable sample measurements, Y, and the respectively evaluated dependent variable, .
In accordance with the present invention, the essential weight factor, , may be defined as comprising a tailored weight factor, W, being multiplied by the square of a deviation normalization coefficient, . The purpose of said deviation normalization coefficient is to render the deviation so as to be characterized by a non-skewed homogeneous uncertainty distribution mapped on to a selected dependent variable coordinate. In accordance with the present invention, said deviation normalization coefficient may be defined as the ratio of a non-skewed dependent component deviation to a dependent coordinate deviation mapping, generally rendered as a presumed skew ratio, , normalized on the square root of said non-skewed dependent component deviation variability, :
                              𝒞          =                      ℛ                                          𝒱                ℛ                                                    ,                                            ℛ              𝒢                                                      𝒱                𝒢                                              .                                    (        9        )            The leads to sign, , suggests one of a plurality of considered representations. The calligraphic  subscript implies application to path-oriented data-point projections. A similarly placed sans serif  subscript would imply application to path coincident deviations. In accordance with the present invention, the skew ratio may be defined as the ratio of a non-skewed representation for dependent component deviation to a respective coordinate representation for a considered reduction deviation. In accordance with the present invention, variability is of broader interpretation than the square of the standard deviation. It is not limited to specifying the mean square deviation but may represent alternate forms of uncertainty, including estimates and measurements, as considered in correspondence with respective data sampling or as associated with considered data point projections; and it may be alternately rendered as a form of dispersion accommodating variability or alternately include the effects of independent measurement error and/or antecedent measurement dispersions; said antecedent measurement dispersions being considered in correspondence with uncertainty in said data sampling or in the representation or mapping of path coincident deviations or path-oriented data-point projections as considered herein, or coordinate oriented data-point projections as previously considered by the present inventor in U.S. Pat. No. 7,107,048 and U.S. patent application Ser. No. 11/266,224, now U.S. Pat. No. 7,383,128. In accordance with the present invention, weight factors, skew ratios, deviation coefficients, and variability should all be considered as functions of the provided data as related to a “hypothetically ideal fitting function” and, as such, they (or successive estimates of the same) should be held constant during minimizing and maximizing procedures associated with forms of calculus of variation which may be implemented for the optimization of fitting parameters.
The deviation variability, , as included in representing tailored and essential weighting of squared deviations, in accordance with the present invention, may be considered in at least two general types, which are herein designated symbolically as:
1. , referring to the considered variability of assumed-to-be non-skewed dependent variable data samples; and
2. , referring to estimates for the considered variability of determined values for the dependent variable as a function of independent variable observation samples.
Referring now to deviation variability type 1 and considering a simple application with errors being limited to the dependent variable, that is: assuming a non-skewed homogeneous error distribution in measurements of the dependent variable, for no errors in the independent variable or independent variables (plural, as the case may be) the variability of the dependent component deviation can be considered equal to the mean square deviations (or square of the standard deviation, σY2 of the dependent variable measurements. The respective essential weight factor may be represented as the tailored weight factor, WYk, normalized on the square of the standard deviation and multiplied by the square of the skew ratio:
                              𝒲                      Y            k                          ⁢                ⁢                                  ⁢                              W                          Y              k                                            σ                          Y              k                        2                          ⁢                              ℛ                          Y              k                        2                    .                                    (        10        )            For this specific application, the skew ratio (being rendered for a homogeneous uncertainty distribution) would be equal to one. The subscripts, Y, which are included on the skew ratio and tailor weight factor, imply that the essential weighting is being tailored to the function  of path coincident devations, Yk−k, whose sample measurements, Yk, as normalized on the local characteristic standard deviations, σYk, are assumed representative of non-skewed error distributions. The deviation variability in Equations 10 is assumed to be represented as the mean squared deviation or the square of the standard deviation. The subscript k designates each single observation comprising the dependent and independent variable sample measurements.
In accordance with the present invention, a representation for essential weight factors with the deviation variability type 1, as considered for weighting of path coincident deviations, may be expressed in a general form by Equations 11.
                                          𝒲                          G              k                                =                                    ℛ                              G                k                            2                        ⁢                                          W                                  G                  k                                                            𝒱                                  G                  k                                                                    ,                            (        11        )            wherein general representation for a mapped observation sample, , is included as a subscript to imply allowance, by weight factor tailoring, for any considered representation, transformation, or mapping of a path coincident deviation onto the currently considered dependent variable coordinate, as a function of −1 independent variables, .
In accordance with the present invention, a representation for essential weight factors with the deviation variability type 2, as considered for weighting of squared path-oriented data-point projections, may be expressed in a general form by Equations 12.
                                          𝒲                          𝒢              k                                =                                    ℛ                              𝒢                k                            2                        ⁢                                          W                                  𝒢                  k                                                            𝒱                                  𝒢                  k                                                                    ,                            (        12        )            wherein general representation for a path designator, , is included as a subscript to imply allowance, by weight factor tailoring, for any considered representation, transformation, or mapping of a path-oriented data point projection onto the currently considered dependent variable coordinate as a function of −1 independent variables, .
Assume a general form for said path designator to be a function of the independent variable or variables, such that:=(1, . . . , , . . . , N−1),  (13)where  is considered, in accordance with the present invention, to represent said general form as the function term of a path-oriented deviation which can, for example, be evaluated in correspondence with data samples, XXik, of said independent variable or variables, i.e.k=(X1k, . . . , Xik, . . . , XN−1,k).  (14)So evaluated, the path designator will establish data-point projections, approximate path coincident deviations, or dependent coordinate mappings of displacements which, when most appropriately rendered, should directly correspond in proportion to the measure between the data point and the intersection of said path with the considered fitting function (or a reliable estimate of the same.)
In accordance with the present invention, the subscript , as considered herein, may be replaced by an alternate subscript, , to distinguish the normalization of path coincident deviations being based upon a concept of sample displacements from true or expected values. Certain past concepts of statistics have been hypothetically based upon this specification. These concepts can only be consistent with Equation 13 provided that the true or expected value can be expressed as a function of orthogonal variable samples. Such cannot be the case when there are errors in said orthogonal variable samples. For appropriate applications, at least one of three alternate considerations can be made:
#1. One can assume that errors in independent variables are indeed small or nonexistent;
#2. For a sufficient amount of data, if the considered path as represented or appropriately weighted can be considered to correspond to a mean deviation path, then one can assume path coincident deviations; or
#3. One can replace the considered residual deviations by dependent coordinate mappings of path-oriented data-point projections between sampled data points and points that lie on the considered fitting function.
Coordinate oriented data-point projections are defined in U.S. Pat. No. 7,107,048 as “ . . . the projection of elements of data-point sets (or data-point defining sets) along corresponding coordinates onto a representation of a data inversion comprising an approximating relationship or a considered estimate of the same.” Path-oriented data-point projections, as disclosed in accordance with the present invention, establish an alternate data reduction concept, which is not limited to orientation along coordinate axes but can be alternately considered along any appropriately characterized deviation path.
Referring to consideration #1, as the errors in the independent variables are small or nonexistent, the independent variable data samples can be considered to lie on the fitting function proper, and the path designator of Equation 13 can be correspondingly evaluated, thus, providing a valid reduction when errors are limited to the dependent variable.
To address consideration #2, that of path coincident deviations, that is, assuming that the defined path might represent a mean deviation path: This assumption has to be based upon the premise that the path designator, as an evaluated function of displaced data samples, is a sufficiently accurate approximation and that the defined deviation path actually represents the expected path of the deviations. In accordance with the present invention, by assuming path coincident deviations, the Gaussian distribution of Equation 1 can be alternately expressed by the approximation of Equation 15 to accommodate maximum likelihood estimating with respect to associated deviation paths with type 1 deviation variability:
                                          D            (                                                            W                  G                                ⁢                                                                            ℛ                      G                      2                                        ⁡                                          (                                              G                        -                        𝒢                                            )                                                        2                                                            2                ⁢                                  𝒱                  G                                                      )                    ≈                                    1                                                2                  ⁢                  π                  ⁢                                                                          ⁢                                      M                    G                                                                        ⁢                          ⅇ                              -                                                                            W                      G                                        ⁢                                                                                            ℛ                          G                                                ⁡                                                  (                                                      g                            -                            𝒢                                                    )                                                                    2                                                                            2                    ⁢                                          𝒱                      G                                        ⁢                                          M                      G                                                                                                          ,                            (        15        )            wherein the deviation is considered as lying along the designated path, originating from the fitting function, and extending to the data sample. Note that the calligraphic subscript  on the variability, weight factors, and skew ratio of Equations 11 has been replaced in Equation 15 by a sans serif  to indicate that the respective weighting and normalization of the considered deviations are assumed for path coincident deviations to be directly, or at least primarily, associated with the observation uncertainty. The deviation variability, , is correspondingly defined, in accordance with the present invention, as the variability which is to be associated with the normalization of respective path coincident deviations. An approximation sign is included in Equation 15 as a result of the approximation that path coincident deviations be represented as a function of unknown true or expected values.
The capital M with the subscript  in Equation 15 represents the mean square deviation of the normalized and weighted path coincident deviations, as evaluated with respect to the determined fitting function or considered approximations of the same. In accordance with the present invention, MG represents a constant value (or proportionality constant) which need not be included nor evaluated to determine maximum likelihood.
By assuming sample observation likelihood probability, to be proportional to the tailored weight factor at each respective function related observation point, and by also assuming a sufficient number of weighted samples to insure that the sum of the weighted deviations is representative of a Gaussian distribution, the associated likelihood estimators, as written to include tailored weighting to accommodate the respective probabilities of observation occurrence for path coincident deviations, can be approximated by Equation 16:
                              L          G                ≈                              ∏                          k              =              1                        K                    ⁢                                          ⁢                                    1                                                2                  ⁢                  π                  ⁢                                                                          ⁢                                      M                    G                                                                        ⁢                                          ⅇ                                  -                                                                                    W                                                  G                          k                                                                    ⁢                                                                                                    ℛ                                                          G                              k                                                        2                                                    ⁡                                                      (                                                          G                              -                              𝒢                                                        )                                                                          k                        2                                                                                    2                      ⁢                                              𝒱                                                  G                          k                                                                    ⁢                                              M                        G                                                                                                        .                                                          (        16        )            Like Equation 15, as considered in accordance with the present invention, forms of Equation 16 can only be considered approximate due to the fact that the mapping of the path/inversion intersection or path descriptor , for path coincident deviations, can be estimated but not actually be evaluated in correspondence with unknown true or expected points assumed to lie on the pre considered fitting function.
In accordance with the preferred embodiment of the present invention, for path coincident deviations, the tailored weight factors, WGk, may be defined as the square root of the sum of the squares of the partial derivatives of each of the independent variables as normalized on square roots of respective local variabilities, or as alternately rendered as locally representative of non-skewed homogeneous error distributions, said partial derivatives being taken with respect to the locally represented path designator  multiplied by a local skew ratio, G, and normalized on the square root of the respectively considered type 1 deviation variability, √{square root over ()}.
                                                                        W                                  G                  k                                            =                                                                    ∑                                          i                      =                      1                                                              N                      -                      1                                                        ⁢                                                            (                                                                                                    ∂                                                          𝒳                              i                                                                                /                                                                                    𝒱                              i                                                                                                                                                            ∂                                                          ℛ                              G                                                                                ⁢                                                      𝒢                            /                                                                                          𝒱                                G                                                                                                                                                        )                                                              𝒫                      ⁢                                                                                          ⁢                      k                                        2                                                                                                                          =                                                                                          𝒱                                              G                        k                                                                                    ℛ                                              G                        k                                            2                                                        ⁢                                                            ∑                                              i                        =                        1                                                                    N                        -                        1                                                              ⁢                                                                  1                                                  𝒱                          ik                                                                    ⁢                                                                        (                                                                                    ∂                                                              𝒳                                i                                                                                                                    ∂                              𝒢                                                                                )                                                                          𝒫                          ⁢                                                                                                          ⁢                          k                                                2                                                                                                                                                    (        17        )            wherein the sans serif subscript, i, implies representation of an independent variable. The k subscript indicates local evaluation or measurement corresponding to an observation comprising N dependent and independent variable sample measurements. In accordance with the present invention, both variability and skew ratio may be assumed to be functions of the observed phenomena as related to an ideal fitting function and associated data sampling and, therefore, considered as observation constants which can be removed from behind and placed in front of the differential sign. In accordance with the present invention, the terminology, as locally representative of a non-skewed homogeneous error distribution, is meant to imply representation as an element of a set or grouping of considered coordinate corresponding observation sample measurements of a same non-skewed homogeneous error distribution.
In accordance with the present invention, the fitting function and respective notation may be arranged to place alternate variables in position to be considered as dependent variables. For example, by replacing the subscript i of Equations 17 with the subscript j, to designate correspondence with both dependent and independent variables in the sum, the tailored weight factor can be alternately written as:
                                                                        W                                  G                  k                                            =                            ⁢                                                                    ∑                                          i                      =                      1                                                              N                      -                      1                                                        ⁢                                                            (                                                                                                    ∂                                                          𝒳                              i                                                                                /                                                                                    𝒱                              i                                                                                                                                                            ∂                                                          ℛ                              G                                                                                ⁢                                                      𝒢                            /                                                                                          𝒱                                𝒢                                                                                                                                                        )                                                              𝒫                      ⁢                                                                                          ⁢                      k                                        2                                                                                                                                        ⁢                            ⁢                                                                                                                  𝒱                                                  G                          d                                                                                            ℛ                                                  G                          d                                                2                                                              [                                                                                                                        -                            1                                                                                𝒱                            d                                                                          ⁢                                                                              (                                                                                          ∂                                                                  𝒳                                  d                                                                                                                            ∂                                                                  𝒢                                  d                                                                                                                      )                                                    2                                                                    +                                                                        ∑                                                      j                            =                            1                                                    N                                                ⁢                                                                              1                                                          𝒱                              j                                                                                ⁢                                                                                    (                                                                                                ∂                                                                      𝒳                                    j                                                                                                                                    ∂                                                                      𝒢                                    d                                                                                                                              )                                                        2                                                                                                                ]                                                        𝒫                    ⁢                                                                                  ⁢                    k                                                                                                          (        18        )            wherein the dependent component is subtracted from the sum. The subscript d is included to designate a specific variable as the dependent variable. The respective path designator, d, and mapped observation sample, , need to be rendered accordingly.
With regard to consideration #3, to accommodate path-oriented data-point projections, one has to re-think the maximum likelihood estimator and establish likelihood as related to the deviation of possible fitting function representations from the observation samples, not as the deviation of the observation samples from unknown expected or true values along the function. With this alternate view of the deviation, in accordance with the preferred embodiment of the present invention, a representation of the respective mapping or path descriptor can be made by successive approximations, and for a deviation variability of type 2, the Gaussian distribution of Equation 1 may be replaced and more appropriately expressed by Equation 19:
                              D          (                                                    W                𝒢                            ⁢                                                                    ℛ                    𝒢                    2                                    ⁡                                      (                                          𝒢                      -                      G                                        )                                                  2                                                    2              ⁢                              𝒱                𝒢                                              )                =                              1                                          2                ⁢                π                ⁢                                                                  ⁢                                  M                  𝒢                                                              ⁢                                    ⅇ                              -                                                                            W                      𝒢                                        ⁢                                                                                            ℛ                          𝒢                          2                                                ⁡                                                  (                                                      𝒢                            -                            G                                                    )                                                                    2                                                                            2                    ⁢                                          𝒱                      𝒢                                        ⁢                                          M                      𝒢                                                                                            .                                              (        19        )            Notice that the subscripts have been switch from what they were in Equation 15, indicating that the deviation variability of the data-point projections, as considered in Equation 19, is related to the independent variable sampling. The respective likelihood estimator can take the considered form of Equation 20,
                              L          𝒢                =                              ∏                          k              =              1                        K                    ⁢                                    1                                                2                  ⁢                  π                  ⁢                                                                          ⁢                                      M                    𝒢                                                                        ⁢                                          ⅇ                                  -                                                                                    W                                                  𝒢                          k                                                                    ⁢                                                                                                    ℛ                                                          𝒢                              k                                                        2                                                    ⁡                                                      (                                                          𝒢                              -                              G                                                        )                                                                          k                        2                                                                                    2                      ⁢                                              𝒱                                                  𝒢                          k                                                                    ⁢                                              M                        𝒢                                                                                                        .                                                          (        20        )            In accordance with the present invention, for path-oriented data-point projections with deviation variability type 2, the tailored weight factors, Wk, may be defined as the square root of the sum of the squares of the partial derivatives of each of the independent variables as normalized on square roots of respective local variabilities, or as alternately rendered as locally representative of non-skewed homogeneous error distributions, said partial derivatives being taken with respect to the locally represented path designator  multiplied by a local skew ratio, , and normalized on the square root of the respectively considered type 2 deviation variability, √{square root over ()}.
                                                                        W                                  𝒢                  k                                            =                            ⁢                                                                    ∑                                          i                      =                      1                                                              N                      -                      1                                                        ⁢                                                            (                                                                                                    ∂                                                          𝒳                              i                                                                                /                                                                                    𝒱                              i                                                                                                                                                            ∂                                                          ℛ                              𝒢                                                                                ⁢                                                      𝒢                            /                                                                                          𝒱                                𝒢                                                                                                                                                        )                                                              𝒫                      ⁢                                                                                          ⁢                      k                                        2                                                                                                                                        ⁢                            ⁢                                                                                          𝒱                                              𝒢                        k                                                                                    ℛ                                              𝒢                        k                                            2                                                        ⁢                                                            ∑                                              i                        =                        1                                                                    N                        -                        1                                                              ⁢                                                                  1                                                  𝒱                          ik                                                                    ⁢                                                                        (                                                                                    ∂                                                              𝒳                                i                                                                                                                    ∂                              𝒢                                                                                )                                                                          𝒫                          ⁢                                                                                                          ⁢                          k                                                2                                                                                                                                                                                  ⁢                            ⁢                                                                                                                  𝒱                                                  𝒢                          d                                                                                            ℛ                                                  𝒢                          d                                                2                                                              [                                                                                                                        -                            1                                                                                𝒱                            d                                                                          ⁢                                                                              (                                                                                          ∂                                                                  𝒳                                  d                                                                                                                            ∂                                                                  𝒢                                  d                                                                                                                      )                                                    2                                                                    +                                                                        ∑                                                      j                            =                            1                                                    N                                                ⁢                                                                              1                                                          𝒱                              j                                                                                ⁢                                                                                    (                                                                                                ∂                                                                      𝒳                                    j                                                                                                                                    ∂                                                                      𝒢                                    d                                                                                                                              )                                                        2                                                                                                                ]                                                        𝒫                    ⁢                                                                                  ⁢                    k                                                                                                          (        21        )            In accordance with the present invention, the respective form for a type 2 essential weight factor (i.e., an essential weight factor rendered to include type 2 deviation variability) may be represented as by Equations 22.
                              𝒲                      𝒢            dk                          ⁢                ⁢                              ℛ                          𝒢              d                                                          𝒱                              𝒢                d                                                    ⁢                                                            [                                                                                                    -                        1                                                                    𝒱                        d                                                              ⁢                                                                  (                                                                              ∂                                                          𝒳                              d                                                                                                            ∂                                                          𝒢                              d                                                                                                      )                                            2                                                        +                                                            ∑                                              j                        =                        1                                            N                                        ⁢                                                                  1                                                  𝒱                          j                                                                    ⁢                                                                        (                                                                                    ∂                                                              𝒳                                j                                                                                                                    ∂                                                              𝒢                                d                                                                                                              )                                                2                                                                                            ]                                            𝒫                ⁢                                                                  ⁢                k                                              .                                    (        22        )            A similarly formulated type 1 essential weight factor (i.e., an essential weight factor rendered to include deviation variability type 1) may be formulated by replacing the type 2 deviation variability, d, in Equations 22 by type 1.
Referring back to both considerations #2 and #3, with regard to the tailoring of weight factors, in accordance with the present invention, respectively rendered deviations may be considered in general forms expressed by Equations 23 for path coincident deviations,δk≈k−k.  (23)or expressed by Equations 24 for path-oriented data-point projections,δk=k−k.  (24)The mapped observation samples, k, as included in Equations 24, may be represented without approximation as a function of both the dependent variable, Xdk, and independent variable data samples, Xik or Xjk, as well as the respectively determined dependent variable measure, k or , e.g.:kk(, X1k, . . . , Xdk, . . . , XN,k),  (25)wherein=k(X1k, . . . , Xik, . . . , XN−1,k).  (26)In accordance with the present invention there may be one or more independent variables (e.g. for a two dimensional system, and for bicoupled variable pairs as might be associated with rendering forms of hierarchical regressions, the value of  in Equations 25 and 26 would be two, providing for one dependent variable and only one independent variable.) Increasing the number of considered dimensions, as designated by the value of , will increase the specified number of independent variables.
Because the path coincident deviations must be considered as an approximation of the deviation of the data sample from an assumed mean point on the fitting function, the validity of Equations 23 depends upon how closely the mapped observation samples, k, can be estimated as a function of sampled data coordinates. In accordance with the present invention, the mapped observation sample may be alternately rendered as a function of pre-estimated fitting parameters and held constant during successive optimizing manipulations. (This restraint upon the mapped observation sample may be more applicable when considering the deviation of the function from the data than when considering the path coincident deviations of the data from an unknown function location.)
In accordance with the present invention, there are at least three differences between the path coincident deviations rendered by Equations 23 and the path-oriented data-point projections as expressed by Equations 24. These are:
1. Because of opposite orientation, i.e. from the fitting function to the data-point v.s. from the data-point to the fitting function, the sign of the deviations is not the same. The path coincident deviations represent an estimate of the deviations of the data from a true or expected value, while the data-point projections represent the deviation of the fitting function from the data point along the projection path. In accordance with the present invention, the directed displacement and associated sign convention may be reversed and alternately included in correspondence with considered convention without affect upon the magnitude or square of the resulting deviations, provided that in considering certain forms of weighted deviations the same convention is maintained throughout the generating of the associated weight factors.2. The dependent variable cannot be evaluated as a function of an unknown true or expected variable, hence for errors-in-variables applications, the path coincident deviations being evaluated with respect to sampled data can only represent an approximation, while the precision of the evaluations of mappings in correspondence with the path/fitting function intersections of respective data-point projections are limited only by analytical representation and computational accuracy.3. The variability of path coincident deviations is determined in correspondence with the considered variability in the deviations of the dependent variable measurements, while the variability of the respective data-point projections will correspond to that of representing the path/fitting function intersection and should be generated as a function of the variability in the deviations of the independent variables.
Note that the only difference between the tailored weight factors, as defined for path coincident deviations by Equations 17 or 18, and the tailored weight factors, as defined for path-oriented data-point projections by Equations 21, is in the representation of the dependent coordinate deviation variability. In accordance with the present invention, for path coincident deviations, said deviation variability, , should be represented by an estimate for a non-skewed variability corresponding to a respective representation for a dependent variable sample. In accordance with the present invention, for data-point projections, said deviation variability, , should be an estimate of the dispersion in a determined value for a representation of a dependent variable with said representation for a dependent variable assumed to be characterized by a non-skewed uncertainty distribution and with said dispersion excluding the direct addition of the variability in said non-skewed representation of the dependent variable.
In accordance with the present invention, normalization of independent variables is not required for the case of non-skewed homogeneous error distributions in respective sample measurements. For completely general application, in accordance with the present invention, the calligraphic  may alternately represent any path designator which is considered typical of a residual, characteristic deviation, or data-point projection, which is assumed, considered, mapped, transformed, or normalized to be represented by a homogeneous non-skewed error distribution or which is assumed, considered, mapped, transformed, or normalized to be represented by a homogeneous non-skewed error distribution when normalized on the square root of a respective dependent coordinate deviation variability, , and/or when multiplied by a considered skew ratio, .
In accordance with the present invention, the implementing of the analytic code of Equations 17, 18, or 21, in the formulating of tailored weight factors, and the implementing of essential weight factors type 2 as exemplified by Equations 22 or essential weight factors type 1, as may be alternately rendered, provide novel weighting of reduction deviations, which may be subject to orthogonal variable uncertainties and/or constraints, including novel weighting of normal deviations and normal data-point projections for errors-in-variables processing and novel weighting for alternately defined deviation paths.
In addition to the concepts heretofore discussed, a major problem that arises with maximum likelihood models is due to the non-orthogonality of solution sets that may be characterized by a fitting function. Unlike orthogonal transforms in which included functions can represent independent components, fitting functions, as considered to be parametric function families, are characterized by fitting parameters which can be represented by a number of evaluation sets, each set appearing to render the fitting function to fit the data, but each set being represented by alternate fitting parameter evaluations. Often the bias of the data or the inclusion of a coordinate offset can prevent conversion or lead to an incorrect evaluation set. To alleviate the problem at least to some degree, in accordance with the present invention, any one or combinations of three alternate approaches might be considered. These are:
1. Bicoupled variable measurements can be considered in hierarchical order, and for many applications, respective bivariate regressions can be rendered.
2. Essential weight factors can be rendered to combine a limited number of squared bivariate reduction deviations in rendering a multivariate sum for the simultaneous evaluating of respective coordinate related fitting parameter estimates.
3. Alternate likelihood estimators can sometimes be combined to add controlling constraints which can filter out at least some of the spurious evaluation sets and allow for more likely representation of an appropriate inversion.
In accordance with the present invention, processing techniques, such as rendering likelihood as related to path-oriented data-point projections as considered herein, as well as alternate schemes for processing path coincident deviations in accordance with the present invention, may be combined by various techniques to provide additional fitting parameter constraints and thus allow for enhanced evaluation.